An omnidirectional camera is known as an apparatus for providing a landscape image around a user. An omnidirectional video image system of the type mentioned is typically formed from a plurality of cameras disposed so as to pickup images around a certain one point in a space determined as a viewpoint. An omnidirectional video image system formed from a plurality of cameras performs an image process of connecting boundaries of picked up images of adjacent ones of the cameras to produce an image over a wide region much greater than the visual field of each of the cameras as an image which looks as if it were picked up by a single wide angle camera.
Although a camera can pickup an image over a wide range if a lens having a wide view angle is used for the camera, the resolution decreases as much and details of a picked up image become less distinguishable. In contrast, where an omnidirectional video image system is used, a picked up image over a wide range can be provided with a high resolution maintained.
Where such an omnidirectional video image as just described is used, an image of the free viewpoint type can be watched. For example, in a television game wherein a character (the cast) can move freely in a space, a background screen from an arbitrary viewpoint can be displayed. Consequently, it becomes possible to enjoy the game through more realistic video image and is augmented in the entertaining degree.
Further, although an omnidirectional video image has a greater capacity compared with an ordinary video image, since it is superior in the interactivity, it is promising as new contents in the broadband network age.
Several proposals have been made for the omnidirectional video image system. For example, in Japanese Patent Laid-Open No. Hei 9-62861, a panorama video image editing apparatus which can display a panorama video image wherein the viewpoint and the direction of the line of sight are continuously changed in response to the designation of the user has been disclosed. The panorama video image editing apparatus disclosed in the publication mentioned synthesizes, based on image data of images picked up around a point which moves drawing an arbitrary locus in a space and is determined as a viewpoint, panorama images from the individual viewpoints and sets successive viewpoint position information to the individual panorama images. Upon reproduction, the panorama video image editing apparatus calculates the position of each display image in the space from the information of the viewpoint position and the direction of the visual line of the display image and information of the directions of movement of the viewpoint and the visual line inputted by the user to synthesize the display images.
However, conventionally the publication mentioned above does not particularly suppose such a situation that the real time performance is required with regard to the synthesis of video images of a high picture quality such as those of a television set as a display apparatus or video images from arbitrary viewpoints. Further, the publication mentioned above has not mentioned about a method of storing omnidirectional video images taking the performance or conditions of the image display or reproduction side. Furthermore, the publication mentioned above does not take into consideration an image storage method which not only views in an arbitrary viewpoint direction from the inner side but also takes utilization from the outside into consideration.